Vehicular driver monitoring system

ABSTRACT

A vehicular driver monitoring system includes a driver status information acquisition system and an in-vehicle control system of the vehicle. The driver status information acquisition system receives data from a plurality of sensors in the vehicle and determines the driver status responsive to processing of the received data. While the driver is not driving the vehicle and the in-vehicle control system is autonomously driving the vehicle, and responsive to determination that the driver of the vehicle should take over driving the vehicle from the in-vehicle control system, the in-vehicle control system alerts the driver to indicate that the driver should take over driving the vehicle. Responsive to the status of the driver while the driver is not driving the vehicle being indicative of the driver not being able to take over driving the vehicle, the in-vehicle control system continues driving the vehicle and initiates an emergency action.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/463,293, filed Mar. 20, 2017, now U.S. Pat. No. 10,703,204,which claims the filing benefits of U.S. provisional applications, Ser.No. 62/414,931, filed Oct. 31, 2016, and Ser. No. 62/312,127, filed Mar.23, 2016, which are hereby incorporated herein by reference in theirentireties.

FIELD OF THE INVENTION

The present invention relates generally to a vehicle vision system for avehicle and, more particularly, to a vehicle vision system that utilizesone or more cameras at a vehicle.

BACKGROUND OF THE INVENTION

Driver and passenger monitoring systems are known. Examples of suchknown systems are described in U.S. Pat. Nos. 7,914,187 and/or6,498,620, which are hereby incorporated herein by reference in theirentireties.

SUMMARY OF THE INVENTION

The present invention provides a driver assistance system that includesa vehicle data acquisition system operable to determine vehicle statusinformation, such as when a vehicle is involved in a collision or whenan autonomous control is ready to hand control of the vehicle back tothe driver. The vehicle status information may include informationpertaining to (i) a collision of the vehicle, (ii) an overturning of thevehicle, (iii) water intrusion into the vehicle and (iv) airbagdeployment. The system includes a driver information acquisition systemoperable to determine health parameters of the driver of the vehicle,and a communication system of the vehicle that is operable tocommunicate with a remote assistance system. Responsive to the vehicledata acquisition system and the driver information system, thecommunication system communicates information to the remote assistancesystem and, responsive to the communication received by the remoteassistance system, the remote assistance system determines anappropriate response to the vehicle collision. For example, if thesystem determines that the vehicle is involved in a collision anddetermines that the driver (or passenger) is badly hurt, the remoteassistance system may send appropriate medical care and personnel to thescene of the accident.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a side elevation of a driver in a driver seat of a vehiclewith a monitoring system that incorporates a camera in accordance withthe present invention;

FIG. 2 shows the COSINUSS® one for measuring body temperature (via aPt1000), heart rate and heart rate variability by optical means andunder use of a three axis acceleration sensor, the device transmits datawirelessly by ANT or Bluetooth smart (4.0);

FIG. 3 is a side elevation of the driver in the driver seat with severalseat head rest, steering wheel and seat belt integrated sensors and anin-cabin camera for detecting the driver's consciousness, health oralive parameter and pose;

FIG. 4 is a plan view of a vehicle compartment with four occupants, eachof the occupants wearing a remotely connected consciousness, health oralive parameter detection devices; and

FIG. 5 is a block diagram of a network structure for processing inputdata of vehicle inherent and occupant-worn consciousness, health oralive parameters, which get processed by structure blocks of multiplealgorithms or systems for assessing each occupant's health condition.

LEGEND TO THE FIGS. 1 TO 5

21 head rest position capacitive sensor

22 belt stretch force sensor

23 belt integrated capacitive cardio sensor

24 belt integrated acceleration sensor

25 seat rest integrated capacitive sensor

26 seat rest integrated acceleration sensor

27 seat bottom integrated capacitive sensor

28 seat bottom integrated acceleration sensor

29 in-cabin driver camera

30 steering wheel touch sensor

31 driver's eye

32 dashboard

33 windshield

34 seat rest integrated thermometer

35 seat bottom integrated thermometer

36 belt integrated thermometer

40 occupant data processing device

41 wireless data transmitter

42 smart watch (with wireless data transmitter)

43 hearing aid (with wireless data transmitter)

44 implantable cardioverter-defibrillator (with wireless datatransmitter)

45 virtual reality googles

46 FIR camera

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A vehicle vision system and/or driver assist system and/or alert systemoperates to determine alertness of the driver and/or health parametersof the driver to determine the severity of injuries to the driver (orpassenger) of a vehicle following a determination of a vehicle collision(or following a determination to hand over driver control to the vehiclethat was previously being autonomously driven).

Live parameter systems are known in medicine and sports equipmentindustry. Nowadays more and more people are wearing fitness detectiondevices (wearables), often worn on the arm or integrated into a watch.Typically, these devices possess wireless communication capabilities.Often these also measure optically by analyzing transmissive orreflected light, for example, for detecting an electrocardiogram (ECG orEKG) measured indirectly by optical means (observation of lightreflected through tissue with blood vessels, since the swallowing of theblood vessel during the heart systole the ‘darker’ vessel reflects lesslight). Often these measure electrical properties of the heart such as,for example, for detecting an ECG, measuring the heart beat by detectingthe heart muscle's electrophysiological pattern of depolarizing duringeach heartbeat. Some of these measure electrical properties of thebrain, such as, for example, for detecting an electroencephalogram(EEG). For non-stationary use especially brain implants became commonwhich measure EEG patterns specifically concerning Parkinson or Epilepsyand encounter the disabilities by emitting electrical brain stimuliitself. Other optical systems are known which monitor the blinking of aproband such as by using webcams. Smartphone apps are offered which aresupposed to detect human body activity and parameters, such as stepcounters, which are measuring velocity and acceleration, where thebouncing caused to the smartphone when a carrying person is running orwalking is determined. Some apps listen to the breathing sound a humancauses and close to the breathing rate by that. Vehicle live parameterdetection systems are known which attempt to detect an occupant's pulseby capacitive elements 21, 23, 25, 27 within the seating structure or byelectrical conducting elements within the steering wheel 30, such asshown exemplary in the FIG. 3. These can also be used for detecting theoccupant's presence and seating pose. Automatically adjusting head restsmay use a capacitive sensor 21 for detecting the head position.

Driver or occupant health or alive parameter supervision may be ofinterest for enabling an autonomous vehicle to hand the vehicle control(back-) over to a human driver in certain situations, especially for SAELevel 3 and 4 autonomous or semi-autonomous vehicles. This can onlyhappen when the driver is able to overtake control of the vehicle in atimely fashion, otherwise the autonomous vehicle has to invoke safetymeasures, such as immediately stopping at the side of the road. Anautonomous or semi-autonomous control system thus may only allow thedriver to take over full driving control of the vehicle when the systemdetermines that the driver's health parameters are indicative of an ableand alert (not disabled or asleep) driver. Optionally, the controlsystem may, responsive to determination of the driver's healthparameters being indicative of a severe or life threatening healthcondition of the driver, may autonomously drive the vehicle to ahospital or medical center or may autonomously wirelessly communicate adistress signal to a remote server to call for help. The system maycommunicate the health parameters to the server or medical center sothat the medical personnel are prepared for the particular health issueof the driver.

Another use case are known automated e-call systems which transmit anemergency call (E-call) after occurrence of a vehicle accident(automatically). Especially the blowing or deployment of the air bags istypically a trigger for activating the automated E-call. E-call systemstypically also transmit the ego vehicle's position.

Driver supervision may be improved according the present invention bycollecting additional occupant constitution, health or alive parameterand E-call systems may be improved by not just transmitting an emergencycall after occurrence of a vehicle accident, but also by collecting andtransmitting additional vehicle parameter and occupant constitution,health or alive parameters, such as (at least one of or more):

-   -   Seat occupancy (before and after crashing)    -   Blown airbags (position)    -   Automated fire extinguish systems have triggered    -   Smoke detector systems have triggered    -   Water intrusion into the cabin    -   Explosive or toxic gas (or smoke) intrusion into or emergence in        the cabin    -   Water, air and ambient temperature    -   Weather conditions    -   Severity of the vehicle accident (by vehicle data record        analysis) or alternatively the vehicle data itself with        executing the severity determination remotely (at ‘call’ the        receiving side).    -   Live parameters of the driver and the other occupants, these may        include:        -   Breathing or breathing rate        -   Heart beat or heart beat rate (ECG data)        -   Blood pressure        -   EEG data        -   Blood glucose level        -   Blood oxygen level        -   Occupant's temperature        -   transpiration        -   pitch of voice (when speaking, crying or groaning)        -   Occupant's drug levels        -   Occupant's alcohol levels

The severity of an accident may be measured by analyzing the data ofvehicle inherent acceleration sensors, accessible via a vehicle bus suchas a CAN bus or the like, and vehicle level (tilt, yaw, roll) that havebeen recorded shortly before a vehicle's accident or impact. The timelyfashion, especially the peaks of acceleration may tell whether or notoccupants may have been injured likely more severe or less severe, andthe more or less a (typically negative) acceleration has been due to thevehicle colliding with another vehicle or object. The vehicle level datamay tell whether a vehicle over turning has taken place. Optionally,overturning and impacting may have happened in combination orconsecutively. Water intrusion may be detected by water intrusionsensors such as capacitive pressure tins or electrical by measuring theelectrical resistance between two nodes typically having a highresistance in air and a lower one when a gap in between is filled bysalt water or freshwater. Water intrusion may point to a severeemergency situation of the occupants, especially when the last positionwas in the near of a waterside. Optionally, the severity of an accidentanalyzation may be done by an artificial intelligence humanhealth-expert system adapted from known human health-expert or speciallytrained for vehicle accident severity analyzation and counter measureinitialization, preferably at the E-call receiving side.

Responsive to a determination of very low severity accidents, a vehicleaccident assistant system may offer to call the police or a tow truckservice instead of an ambulance. Optionally, the recording may be doneby a black box or occupant data processing device or E-call device 41which exceeds the usual amount of data that black box devices usuallystore due to recording additional occupant constitution, health or aliveparameters as discussed above (See FIG. 4). Similar to E-call devices,the device 41 may include long range radio transmission means, such asmeans comprising the 5G standard.

By collecting and transmitting one or more of the above said health oralive parameters, especially consciousness, breathing and heartbeat, the(receiving) E-call service station may be able to assess the emergencysituation better than without. The service may know better how manyambulances or rescue helicopters it should send to the accident site andhow many and which kind of and ambulance personnel is/are required andwhich experience level these should have. The hospitals to whichemergency patients may be brought may be prepared better as well, byhaving a longer preparation time and having advance knowledge as towhich kind of injuries the known number of emergency patients will have.Optionally, traffic control means or services may also benefit by havingbetter knowledge of the severity at accident sites, especially whenplenty of the vehicles are equipped with these health or alive parametercollection and transmission systems in accordance with the presentinvention.

Breathing or breathing rate may be measured or determined within thevehicle inherent health or alive parameter collection system by using incabin cameras 29 (see FIGS. 1, 3 and 4). The breathing may be detectedby supervision of the upper body raising and lowering. Optionally, thebreathing may be measured by a stretch force sensor 22 integrated intothe seatbelt (see FIG. 3). Optionally, the stretch force sensor data maybe fused with visual data. Optionally, an upper body volume calculatingmodel may find use. During inhaling the volume should increase andduring exhaling the volume should decrease and the force to the belt mayincrease during inhaling and decrease during exhaling. Since the humanbreath rate is in the area of 40 to 300 per minute the force and volumeerror noise can be band filtered to this cycle band or to 0 . . . 300 todetect the stopping of breathing well.

Alternatively or additionally, the breathing may be detected bydetecting the nicking of the head during breathing. The head nicking canalso indicate to the heart beating over time. The heart beat mayalternatively be visible or detectable at the throats veins. Optionally,the skin's color change during the systole to more reddish and back tomore yellowish during the diastole may be used for detecting the heartbeat as additional or alternative visual detection method.Alternatively, the blood swooshing through veins and arteries may bedetectable acoustically by microphones or acceleration sensors, whenfiltering other noise. This may work when close in contact to the human,such as when the microphones or acceleration sensors are integratedwithin the seat structures 26, 28 or the steering wheel 30 or seat belt24. The heart beat may be low pass filtered in an area of 0 . . . 150per minute. Optionally, different measuring methods may be used incombination and optionally may be specially trained to each occupant'sspecific measuring patterns within an artificial intelligence algorithmand memory. Optionally, one or more of the sensors in the seat belt orseat back or seat or head rest may comprise a radar sensor that monitorsand detects the seat occupant's respiration and/or heart rate.

Optionally, the occupant's or occupants' temperature may be measured byone or more thermometers integrated into the seat structure 34, 35.Optionally, a thermometer 36 may be integrated into the seat belt eitherat the surface pointing to the [individual] occupant's chest or weavedwithin the seat belt. Optionally, the occupants' temperature may bemeasured by another far infrared (FIR) sensor or camera 46 or anothercamera FIR-imager within the RGB camera 29 directed to a skin surface ofthe according occupants, such as the hands or faces (see FIG. 4).

Optionally, one or more microphones (FIG. 4) may detect the occupants'voices. The pitch of the occupants' voices may point to theirconstitution. At least the fact that the occupants are speaking, cryingor groaning may indicate to their consciousness. There may be vehicleinherent microphones to detect the occupants' voices. Optionally, themicrophones may be part of a vehicle noise suppression and voiceamplification system or sound processing system, similar to onesdescribed in and utilizing aspects of the systems in U.S. PublicationNo. US-2016-0029111, which is hereby incorporated herein by reference inits entirety.

Optionally, additionally or instead of using vehicle inherent sensors orsensor systems for collecting driver or occupant health or aliveparameters, the vehicle system according the invention may be made in away to receive health or alive data from non-vehicle inherent sensorsdirect or wirelessly such as from:

-   -   smart phones;    -   smart watches (42);    -   smart goggles (also called smart glasses) (45);    -   smart piercings;    -   smart ear rings;    -   smart necklace or other smart jewelry;    -   smart hearing aid devices (43);    -   Electronic foot cuffs (for prisoners on parole);    -   fitness detection devices (wearables such as, for example,        breast belts, wristlets or (sport-) earphones); or    -   occupant's body implants or body sensors with wireless        communication capabilities such as:        -   Ingestible sensors (sensing the diet or the eupepsia) and            actors (cyber pills);        -   Stomach patches (sensing the diet, tracks medication taking,            steps, activity, rest and heart rate);        -   Implant for gastric stimulation (IGS) (against            Gastroparesis);        -   Tracking implants (for example via RFID);        -   Cochlea implants;        -   Retina implants;        -   Blood glucose dosimeter implants;        -   Smart (electronic controlled) drug release/dosimeter implant            or stent;        -   Smart (electronic controlled) Enzyme or Hormone            release/dosimeter implant (for example for birth control);        -   Smart (electronic controlled) bacteria, virus, prion, or            bacteriophage release/dosimeter implant (for example as used            for encountering brain tumors by genetic altered            salmonella);        -   Smart (electronic controlled) sphincter prosthesis;        -   Smart teeth or tooth inlay or implant;        -   Blood pressure measuring implants;        -   Brain pacemaker devices (such as against Parkinson or            Epilepsy);        -   Brain-machine interfaces (e.g., for brain-remote-controlled            machine functions);        -   Nano bots (for example injected into the blood stream);        -   Diaphragm pacemakers (by electrical stimulation of a phrenic            nerve);        -   Cardiac pacemaker devices;        -   Defibrillation implants (implantable            cardioverter-defibrillator (ICD))(44);        -   Artificial organs (for example heart-pumps, heart valves,            kidney machine);        -   Smart (electronic controlled) prosthesis for treating            disabilities (for example knee prosthesis with a smart            algorithm for controlling the bending according the walkers            pace); and/or        -   Smart (electronic controlled) surgical prosthesis (for            example prosthesis for enlarging the legs by Distraction            osteogenesis (DO) known as FITBONE®).

In FIG. 4, each of the occupants wears one of the above remotelyconnected conscious, health or alive parameter detection devices. Thewireless connection may be comprised optically, inductively or viaanalog or digital radio transmission of any kind via a vehicle inherentwireless data transmitter 41 or a vehicle attached (non-inherent) devicesuch as a smart phone with blue tooth connection to the above wirelessdevices. The above non implants such as the smart watch may collecthealth or alive parameter via skin contact, measuring electrical fieldsor body resistance, or may measure optically, such as by detecting thetransmissivity of the skin. Optionally, the non-implants may measure thebreathing and/or the blood flow acoustically via a microphone attachedto the wearer's skin. As one example, shown in FIG. 2, the COSINUSS® Onefrom Cosinuss GmbH, Kistlerhofstr. 60, D-81379 Munich, may be used. Theear phone like device measures the heart beat via light transmissivity,electrically and via an acceleration sensor in combination in the earcanal, additionally it measures the body temperature and the oxygenlevel which gets send via BLUETOOTH or ANT+ to a fitness watch or asmart phone. This type of device collects body data that may betransmitted to the vehicle occupant data processing device 40 via awireless transmitter 40 of any kind.

Optionally, the driver or vehicle occupants may have microchip implantswhich may be made for the purpose of transmitting health or aliveparameters intentionally or as a by-product functionality servingadditional main purposes such as fitness tracking, drug supervision orpositional tracking. For example, it may appear that intelligenceagents, soldiers or prisoners on day parole may have such positionaltracking microchips implanted already today. These may become morewidespread with increasing user benefit. These may be similar to theimplant microchips known today for implanting in pets for identifyingthe owner when the pet is found. Typically, these work with passive RFIDor near field communication (NFC). Optionally, these may have activeradio data communication systems with longer range, such as wirelessLAN, BLUETOOTH or active RFID comprised on the microchip implant, or thelong range transmitter may optionally be outside of the body, such as,for example, as part of a smart phone, for further routing bodyparameter signals received from the implant having a short rangeconnection such as RFID, BLUETOOTH or NFC to the smart phone.

Optionally, vehicle inherent data transmission devices may receive thesebody implants data directly, such as via a NFC connection with theantenna in the steering wheel hub. Optionally, the non-vehicle inherenthealth or alive parameter transmitting device or implant may run onenergy harvesting, such as by harvesting energy form the likes of radiowaves (inductive), vibration (via a piezo), optical (via photovoltaics),temperature differences (via a Seebeck element), chemical reaction(e.g., by reduction of blood sugar), biological (e.g., by energyharvesting bacteria, fungi, lichen, algae or plants).

The data may be collected and optionally transmitted singly oralternatively combined with other non-vehicle inherent sensor dataand/or combined with other vehicle inherent sensor data. Optionally,different measuring methods and measuring sensor source data may be usedin combination and optionally may be specially trained to eachoccupant's specific measuring patterns within an artificial intelligencealgorithm (AI). Optionally, the AI's detection output may be aimed foreither determining a driver's driving task take over ability or, for anE-call service, aimed to assess the severity of an emergency situationthat the vehicle occupants may be in.

Different systems may be specialized to treat the specific sensors orgroup of sensors for monitoring the driver health. Optionally, themonitoring of blood pressure and heart rate may be done with neuralnetwork random forest algorithm. Other biological parameters like EEG,ECG or the implantable device data may be monitored with neural networkssuch as deep belief network (deep neural network, can be viewed asrestricted Boltzmann machine). The driver gait analysis as well asothers biometric identifications may be monitored similarly to what istested in hospital for medical diagnostics with a support vector machinealgorithm. Another machine learning algorithm (one of the previouslyevoked or convolutional neural network) should take all the results ofthe different specialized systems to take the decision of the vitalityof each occupant. In FIG. 5, an example of a network structure is shownfor processing a bunch of exemplary input data which are processed inexemplary different kind of artificial intelligence network structureblocks for assessing each occupant's health condition (as output).

Optionally, the E-call service and the driver health or alive parametersupervision system or task, semaphore or application or app may beseparate or combined into one. The system may run on a vehicle bodycontroller or domain main sharing the processing power with othersystems, tasks, semaphores, applications or apps, or may run on an extravehicle control device with processors and/or equivalents, such asFPGAs, DSPs, ASICs and/or the like. Also, the data memory may be holddistributed or cloud based or in one or several of the linked devices.Optionally, the system may run on distributed devices within the egovehicle. Optionally, the system may run in a distributed manner onmultiple devices which are partially vehicle inherent and partially oninherent (mobile), such as running partially on a vehicle bodycontroller and on a smartphone's app. Optionally, as share may run ascloud computing shared with other vehicles' controls or vehicleoccupants' smartphones. Optionally, the system may run shared withremote control services processing devices, such as servers of remotecontrol services, such as discussed below.

In case a driver does not take over the steering wheel after theautonomous vehicle driving system has requested the driver to take overor the driver health or alive parameter supervision system has detectedthe driver is unconscious or non-responsive or unable to take over thevehicle control, a human service person at a (emergency) call center mayget permission and request to take the remote control over the vehiclewith a non-responsive or incapacitated driver. The remote control may bedone by a state of the art wireless radio control with the leastpossible latency. The vehicle inherent ADAS or autonomous drivingfeature may optionally support the emergency service remote controlservice algorithm or preferred remote control service person bytransmitting the vehicle position and environmental sensor data,especially the forward vision data (when the car is just drivingforwardly). Due to latency, the remote control may be in a partiallyvehicle automated and partial human controlled manner. For example, theremote control service person may select a safe spot for the vehicle inemergency to come to a stop during the vehicle may enter the spot in anautomated manner. This may include lane change maneuvers, accelerationand braking actions, such as may be done automated or autonomously. Theservice personal or algorithm may additionally have control over thepotentially present traffic control system. He, she or it may close alane or may engage a warning sign or speed limit for that and adjacentlanes. Optionally, a specific ‘Driver in emergency’ sign may be shown tothe other vehicles' drivers, to raise their attention to let the vehiclein emergency through to its designated safe stopping point. The vehiclein emergency may automatically engage its warning blinkers andoptionally other warning means, such as a text banner display or warningsign triangle that optionally may be displayed on the rear window orother windows.

Thus, the system of the present invention monitors information derivedfrom multiple vehicle and driver and occupant sensors or inputs todetermine a severity of an accident or collision of the vehicle and aseverity of injuries to the driver or passenger(s) of the vehicle. Thesystem may include various sensors in the vehicle (such as microphonesensors and image sensors and infrared sensors and laser optic sensors(such as of the types described in U.S. Publication No. US-2016-0267911,which is hereby incorporated herein by reference in its entirety) or thelike), and may receive inputs or information from various other devices(such as a passenger's smart phone or fitness device or the like), andresponsive to the various sensor data and inputs, the system determinesthe severity of a collision and/or the severity of injuries to one ormore occupants of the crashed vehicle. The monitoring systems or driverinformation acquisition system may utilize aspects of the monitoringsystems described in U.S. Pat. Nos. 8,258,932; 6,166,625 and/or6,485,081, and/or U.S. Publication Nos. US-2015-0296135 and/orUS-2015-0294169, which are hereby incorporated herein by reference intheir entireties. Optionally, the sensing system may utilize aspects ofthe systems described in U.S. Pat. Nos. 8,027,029; 8,013,780; 6,825,455;7,053,357; 7,408,627; 7,405,812; 7,379,163; 7,379,100; 7,375,803;7,352,454; 7,340,077; 7,321,111; 7,310,431; 7,283,213; 7,212,663;7,203,356; 7,176,438; 7,157,685; 6,919,549; 6,906,793; 6,876,775;6,710,770; 6,690,354; 6,678,039; 6,674,895 and/or 6,587,186, and/orInternational Publication No. WO 2011/090484 and/or U.S. Publication No.US-2010-0245066 and/or U.S. patent application Ser. No. 15/420,238,filed Jan. 31, 2017 and published as U.S. Publication No.US-2017-0222311, which are hereby incorporated herein by reference intheir entireties.

The wireless communications may utilize aspects of the systems describedin U.S. Pat. No. 7,580,795 and/or U.S. Publication Nos. US-2015-0251599;US-2015-0158499; US-2015-0124096; US-2014-0375476; US-2014-0218529;US-2013-0222592; US-2012-0218412 and/or US-2012-0062743, which arehereby incorporated herein by reference in their entireties.

The system may also communicate with other systems, such as via avehicle-to-vehicle communication system or a vehicle-to-infrastructurecommunication system or the like. Such car-to-car or vehicle-to-vehicle(V2V) and vehicle-to-infrastructure (car2X or V2X or V2I or 4G or 5G)technology provides for communication between vehicles and/orinfrastructure based on information provided by one or more vehiclesand/or information provided by a remote server or the like. Such vehiclecommunication systems may utilize aspects of the systems described inU.S. Pat. Nos. 6,690,268; 6,693,517; 7,156,796 and/or 7,580,795, and/orU.S. Publication Nos. US-2012-0218412, US-2012-0062743, US-2015-0158499;US-2015-0124096; US-2015-0352953; US-2016-0036917 and/orUS-2016-0210853, which are hereby incorporated herein by reference intheir entireties.

The camera or sensor may comprise any suitable camera or sensor.Optionally, the camera may comprise a “smart camera” that includes theimaging sensor array and associated circuitry and image processingcircuitry and electrical connectors and the like as part of a cameramodule, such as by utilizing aspects of the vision systems described inInternational Publication Nos. WO 2013/081984 and/or WO 2013/081985,which are hereby incorporated herein by reference in their entireties.

The system includes an image processor operable to process image datacaptured by the camera or cameras, such as for detecting objects orother vehicles or pedestrians or the like in the field of view of one ormore of the cameras. For example, the image processor may comprise animage processing chip selected from the EYEQ family of image processingchips available from Mobileye Vision Technologies Ltd. of Jerusalem,Israel, and may include object detection software (such as the typesdescribed in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, whichare hereby incorporated herein by reference in their entireties), andmay analyze image data to detect vehicles and/or other objects.Responsive to such image processing, and when an object or other vehicleis detected, the system may generate an alert to the driver of thevehicle and/or may generate an overlay at the displayed image tohighlight or enhance display of the detected object or vehicle, in orderto enhance the driver's awareness of the detected object or vehicle orhazardous condition during a driving maneuver of the equipped vehicle.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ladar sensors or ultrasonicsensors or the like. The imaging sensor or camera may capture image datafor image processing and may comprise any suitable camera or sensingdevice, such as, for example, a two dimensional array of a plurality ofphotosensor elements arranged in at least 640 columns and 480 rows (atleast a 640×480 imaging array, such as a megapixel imaging array or thelike), with a respective lens focusing images onto respective portionsof the array. The photosensor array may comprise a plurality ofphotosensor elements arranged in a photosensor array having rows andcolumns. Preferably, the imaging array has at least 300,000 photosensorelements or pixels, more preferably at least 500,000 photosensorelements or pixels and more preferably at least 1 million photosensorelements or pixels. The imaging array may capture color image data, suchas via spectral filtering at the array, such as via an RGB (red, greenand blue) filter or via a red/red complement filter or such as via anRCC (red, clear, clear) filter or the like. The logic and controlcircuit of the imaging sensor may function in any known manner, and theimage processing and algorithmic processing may comprise any suitablemeans for processing the images and/or image data.

For example, the vision system and/or processing and/or camera and/orcircuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641;9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401;9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169;8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935;6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229;7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287;5,929,786 and/or 5,786,772, and/or U.S. Publication Nos. 2014/0340510;2014/0313339; 2014/0347486; 2014/0320658; 2014/0336876; 2014/0307095;2014/0327774; 2014/0327772; 2014/0320636; 2014/0293057; 2014/0309884;2014/0226012; 2014/0293042; 2014/0218535; 2014/0218535; 2014/0247354;2014/0247355; 2014/0247352; 2014/0232869; 2014/0218529; 2014/0211009;2014/0160276; 2014/0168437; 2014/0168415; 2014/0160291; 2014/0152825;2014/0139676; 2014/0138140; 2014/0104426; 2014/0098229; 2014/0085472;2014/0067206; 2014/0049646; 2014/0052340; 2014/0025240; 2014/0028852;2014/005907; 2013/0314503; 2013/0298866; 2013/0222593; 2013/0300869;2013/0278769; 2013/0258077; 2013/0258077; 2013/0242099; 2013/0222592;2013/0215271; 2013/0141578 and/or 2013/0002873, which are all herebyincorporated herein by reference in their entireties. The system maycommunicate with other communication systems via any suitable means,such as by utilizing aspects of the systems described in InternationalPublication Nos. WO 2010/144900; WO 2013/043661 and/or WO 2013/081985,and/or U.S. Pat. No. 9,126,525, which are hereby incorporated herein byreference in their entireties.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

1. A vehicular driver monitoring system, said vehicular drivermonitoring system comprising: a driver status information acquisitionsystem operable to determine status of a driver of a vehicle; wherein anin-vehicle control system of the vehicle is operable to autonomouslydrive the vehicle along a road; wherein said driver status informationacquisition system receives data from a plurality of sensors in thevehicle and determines the driver status responsive to processing of thereceived data; wherein said plurality of sensors comprises at least onecamera viewing the face of the driver of the vehicle; wherein, while thedriver is not driving the vehicle and the in-vehicle control system isautonomously driving the vehicle, and responsive to determination thatthe driver of the vehicle should take over driving the vehicle from thein-vehicle control system, the in-vehicle control system alerts thedriver to indicate that the driver should take over driving the vehicle;and wherein, while the driver is not driving the vehicle and thein-vehicle control system is autonomously driving the vehicle, andresponsive to determination that the driver of the vehicle should takeover driving the vehicle from the in-vehicle control system, andresponsive to the status of the driver while the driver is not drivingthe vehicle as determined by said driver status information acquisitionsystem being indicative of the driver not being able to take overdriving the vehicle from the in-vehicle control system, the in-vehiclecontrol system continues driving the vehicle and initiates an emergencyaction.
 2. The vehicular driver monitoring system of claim 1, whereinthe emergency action comprises driving the vehicle to a safe locationand stopping the vehicle at the safe location.
 3. The vehicular drivermonitoring system of claim 1, wherein the emergency action comprisescalling an emergency call center.
 4. The vehicular driver monitoringsystem of claim 1, wherein the emergency action comprises alertingdrivers of other vehicles.
 5. The vehicular driver monitoring system ofclaim 1, wherein the emergency action comprises driving the vehicle to amedical facility.
 6. The vehicular driver monitoring system of claim 1,wherein said driver status information acquisition system receivesdriver status data from at least one selected from the group consistingof (i) a smart phone disposed in the vehicle, (ii) a smart watch of thedriver of the vehicle and (iii) a health monitoring device disposed inthe vehicle.
 7. The vehicular driver monitoring system of claim 1,wherein said driver status information acquisition system receivesdriver status data including at least one selected from the groupconsisting of (i) breathing rate of the driver, (ii) heart rate of thedriver, (iii) blood pressure of the driver, (iv) EEG data of the driver,(v) blood glucose level of the driver and (vi) body temperature of thedriver.
 8. The vehicular driver monitoring system of claim 1, whereinthe in-vehicle control system is responsive at least in part to thedriver status determined by processing image data captured by saidcamera of said driver status information acquisition system.
 9. Thevehicular driver monitoring system of claim 1, wherein the in-vehiclecontrol system, while the driver is not driving the vehicle and thein-vehicle control system is autonomously driving the vehicle, andresponsive at least in part to the driver status being indicative of adisabled driver, will not allow the driver to take over driving of thevehicle from the in-vehicle control system.
 10. The vehicular drivermonitoring system of claim 9, wherein, responsive at least in part tothe driver status being indicative of a disabled driver with a healthcondition, the in-vehicle control system drives the vehicle to a medicalcenter.
 11. The vehicular driver monitoring system of claim 1, whereinsaid plurality of sensors comprises at least one radar sensor sensing anarea occupied by the driver of the vehicle.
 12. The vehicular drivermonitoring system of claim 1, wherein said plurality of sensorscomprises at least one sensor disposed at a seat belt of a driver seatof the vehicle, and wherein said at least one sensor comprises a beltstretch force sensor disposed at the seat belt.
 13. The vehicular drivermonitoring system of claim 1, wherein said plurality of sensorscomprises at least one sensor disposed at a seat belt of a driver seatof the vehicle, and wherein said at least one sensor comprises anacceleration sensor disposed at the seat belt.
 14. The vehicular drivermonitoring system of claim 1, wherein said plurality of sensorscomprises at least one sensor integrated in a driver seat of thevehicle.
 15. The vehicular driver monitoring system of claim 1, whereinsaid plurality of sensors comprises at least one sensor integrated in aheadrest of a driver seat of the vehicle.
 16. A vehicular drivermonitoring system, said vehicular driver monitoring system comprising: adriver status information acquisition system operable to determinestatus of a driver of a vehicle; wherein an in-vehicle control system ofthe vehicle is operable to autonomously drive the vehicle along a road;wherein said driver status information acquisition system receives datafrom a plurality of sensors in the vehicle and determines the driverstatus responsive to processing of the received data; wherein saidplurality of sensors comprises at least one camera viewing the face ofthe driver of the vehicle; wherein said driver status informationacquisition system receives driver status data including at least oneselected from the group consisting of (i) breathing rate of the driver,(ii) heart rate of the driver, (iii) blood pressure of the driver, (iv)EEG data of the driver, (v) blood glucose level of the driver and (vi)body temperature of the driver; wherein, while the driver is not drivingthe vehicle and the in-vehicle control system is autonomously drivingthe vehicle, and responsive to determination that the driver of thevehicle should take over driving the vehicle from the in-vehicle controlsystem, the in-vehicle control system alerts the driver to indicate thatthe driver should take over driving the vehicle; wherein, while thedriver is not driving the vehicle and the in-vehicle control system isautonomously driving the vehicle, and responsive to determination thatthe driver of the vehicle should take over driving the vehicle from thein-vehicle control system, and responsive to the status of the driverwhile the driver is not driving the vehicle as determined by said driverstatus information acquisition system being indicative of the driver notbeing able to take over driving the vehicle from the in-vehicle controlsystem, the in-vehicle control system continues driving the vehicle andinitiates an emergency action; and wherein the in-vehicle controlsystem, while the driver is not driving the vehicle and the in-vehiclecontrol system is autonomously driving the vehicle, and responsive atleast in part to the driver status being indicative of a disableddriver, will not allow the driver to take over driving of the vehiclefrom the in-vehicle control system.
 17. The vehicular driver monitoringsystem of claim 16, wherein the emergency action comprises driving thevehicle to a safe location and stopping the vehicle at the safelocation.
 18. The vehicular driver monitoring system of claim 16,wherein the emergency action comprises calling an emergency call center.19. The vehicular driver monitoring system of claim 16, wherein theemergency action comprises alerting drivers of other vehicles.
 20. Thevehicular driver monitoring system of claim 16, wherein the emergencyaction comprises driving the vehicle to a medical facility.
 21. Thevehicular driver monitoring system of claim 16, wherein, responsive atleast in part to the driver status being indicative of a disabled driverwith a health condition, the in-vehicle control system drives thevehicle to a medical center.
 22. A vehicular driver monitoring system,said vehicular driver monitoring system comprising: a driver statusinformation acquisition system operable to determine status of a driverof a vehicle; wherein an in-vehicle control system of the vehicle isoperable to autonomously drive the vehicle along a road; wherein saiddriver status information acquisition system receives data from aplurality of sensors in the vehicle and determines the driver statusresponsive to processing of the received data; wherein said plurality ofsensors comprises at least one camera viewing the face of the driver ofthe vehicle; wherein the in-vehicle control system is responsive atleast in part to the driver status determined by processing image datacaptured by said camera of said driver status information acquisitionsystem; wherein, while the driver is not driving the vehicle and thein-vehicle control system is autonomously driving the vehicle, andresponsive to determination that the driver of the vehicle should takeover driving the vehicle from the in-vehicle control system, thein-vehicle control system alerts the driver to indicate that the drivershould take over driving the vehicle; wherein, while the driver is notdriving the vehicle and the in-vehicle control system is autonomouslydriving the vehicle, and responsive to determination that the driver ofthe vehicle should take over driving the vehicle from the in-vehiclecontrol system, and responsive to the status of the driver while thedriver is not driving the vehicle as determined by said driver statusinformation acquisition system being indicative of the driver not beingable to take over driving the vehicle from the in-vehicle controlsystem, the in-vehicle control system continues driving the vehicle andinitiates an emergency action; and wherein the emergency actioncomprises at least one selected from the group consisting of (i) drivingthe vehicle to a safe location, (ii) calling an emergency call centerand (iii) alerting drivers of other vehicles.
 23. The vehicular drivermonitoring system of claim 22, wherein the emergency action comprisesdriving the vehicle to a safe location and stopping the vehicle at thesafe location.
 24. The vehicular driver monitoring system of claim 22,wherein the emergency action comprises calling an emergency call center.25. The vehicular driver monitoring system of claim 22, wherein theemergency action comprises alerting drivers of other vehicles.
 26. Thevehicular driver monitoring system of claim 22, wherein the emergencyaction comprises driving the vehicle to a medical facility.